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Abstract — We study ensembles of codes on graphs (generalized 
low-density parity-check, or LDPC codes) constructed from ran- 
dom graphs and fixed local constrained codes, and their extension 
to codes on hypergraphs. It is known that the average minimum 
distance of codes in these ensembles grows linearly with the code 
length. We show that these codes can correct a linearly growing 
number of errors under simple iterative decoding algorithms. 
In particular, we show that this property extends to codes 
constructed by parallel concatenation of Hamming codes and 
other codes with small minimum distance. Previously known 
results that proved this property for graph codes relied on graph 
expansion and required the choice of local codes with large 
distance relative to their length. 

Index Terms — Graph codes, hypergraph codes, iterative decod- 
ing, parallel concatenation of codes. 

I. Introduction 

Considerable attention in recent years was devoted to the 
study of error correction with codes on graphs. In this paper 
we are interested in estimating the number of errors correctable 
with codes on graphs constructed as generalizations of LDPC 
codes. LDPC codes are constructed on a bipartite graph 
G{V,E),V — Vi U V2 by associating code's coordinates 
with the vertices in one part of G, replicating the values 
of each vertex on the edges incident to it, and imposing a 
parity-check constraint at each vertex of the other part of G. 
The generalization that we have in mind is concerned with 
replacing the repetition and single-parity-check codes as local 
codes at the graph's vertices with other error-correcting codes. 

Error correction with codes on graphs has been studied 
along two lines, namely, by computing the average number 
of errors correctable with some decoding algorithm by codes 
from a certain random ensemble of graph codes, or by ex- 
amining explicit code families whose construction involves 
graphs with a large spectral gap. The first direction originates 
in the works of Gallager Q and Zyablov and Pinsker ifTSl 
who showed that random LDPC codes of growing length 
can correct a nonvanishing fraction of errors. Recently the 
decoding algorithm of ifTSl was studied by Burshtein 161 who 
derived an improved estimate of the number of correctable 
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errors compared to (15] and by Zyablov et al. fT4\ who pro- 
vided estimates of the number of errors under the assumption 
of local single error-correcting (Hamming) codes. The second 
line of work, initiated in Tanner's paper (VI] and in Sipser 
and Spielman's [ 10], pursues estimates of error correction with 
codes on regular graphs with a small second eigenvalue and 
ensuing expansion properties. Presently it is known that such 
codes under iterative decoding can correct the number of errors 
equal to a half of the designed distance of graph codes 
This estimate fits in a series of analogous results for various 
"concatenated" coding schemes and has prompted a view of 
graph codes as parallel concatenations of the local codes ||2l. 
However, this result relies on certain restrictive assumptions 
discussed below. 

An extension of Tanner's construction from graphs to hyper- 
graphs was proposed by Bilu and Hoory 14] who showed that 
such codes (for high code rates) can have minimum distance 
greater than the best known bipartite-graph constructions. 
Interestingly, the codes considered in |4| are a direct extension 
of a construction in |7| in the same way as Tanner's graph 
codes extend LDPC codes. 

As is well known, graphs with high expansion and random 
graphs share many properties that can be used to prove esti- 
mates of error correction. This similarity in the coding theory 
context was emphasized in our recent work |T| where we 
showed that ensembles of codes on random graphs and explicit 
expander-like constructions share many common features such 
as properties of the minimum distance and weight distribution. 

Regarding the proportion of errors corrected by graph codes 
under iterative decoding, we note one difference between 
(generalized) LDPC codes on random graphs and explicit 
constructions based on the graph spectrum. The explicit con- 
structions based on regular graphs depend on the difference 
between the largest and the second largest eigenvalue of the 
graph (the "spectral gap"). For this reason, one is forced to 
rely on local codes with rather large minimum distance do, 
for instance, do greater than the square root of the degree 
n of the graph. Even though in the construction of ifTOl and 
later works n is kept constant, this effectively rules out of 
consideration local codes with small minimum distance such 
as the Hamming codes and the like. The square root restriction 
is implied by the spectral gap of regular bipartite graphs, and 
is the best possible owing to the Alon-Boppana bound for 
graph spectra |9|. The purpose of the present work is to lift 
this limitation on the distance do by switching from graphs 
with a large spectral gap to random graphs. 

In this paper we obtain new estimates of the number of 
correctable errors for random ensembles of bipartite-graph and 
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hypergraph codes under iterative decoding. The first part of 
the paper is devoted to codes on regular bipartite graphs. To 
construct long graph codes, we assume that the degree of 
the graph is fixed and the number of vertices in both parts 
approaches infinity. Assuming that local constraint codes are 
used to correct 2 or more errors, we show that almost all codes 
in the ensemble of graph codes are capable of correcting all 
error patterns of weight that forms a constant fraction of the 
code length. This is a much less restrictive assumption on the 
local codes than the one taken in earlier works on decoding of 
graph codes IS), lfT3l . The proof of this result employs some 
ideas of H] introduced there for the analysis of the weight 
distribution of graph codes. 

We then observe that if the degree of the graph is allowed to 
increase then graph codes with local codes of constant distance 
do not correct a linearly growing number of errors under the 
proposed iterative decoding. This motivates us to study graph 
codes with long local codes correcting a growing number of 
errors that forms a fixed proportion of the degree. The results 
obtained in this case parallel earlier theorems for product codes 
and graph codes based on the spectral gap. 

In the second part of the paper we establish similar results 
for codes on hypergraphs, showing that a constant proportion 
of errors is corrected by an iterative decoding algorithm 
that combines some ideas of [1| with the results proved 
for codes on bipartite graphs in the first part of the paper 
Constructing the code ensemble based on regular hypergraphs 
of a fixed degree, we show that they contain codes capable of 
correcting a constant proportion of errors. The proof involves 
no assumptions on the distance of the local codes; in particular, 
we show that networks of Hamming codes correct a fixed 
proportion of errors under iterative decoding. This fact was 
previously proved by Tanner |12| under the assumption that 
the underlying graph is a tree. This assumption is not needed 
in our results. As in the case of the graph ensemble, we also 
perform the analysis of the decoding algorithm for the case 
of growing degree, finding the proportion of errors correctable 
with hypergraph codes based on long local codes. 

This paper is dedicated to the memory of Ralf Koetter. The 
first-named author discussed the problem of estimating the 
performance of codes on graphs with Ralf in the beginning 
of 2004. Ralf 's idea at that point was to investigate the error 
correcting capability of codes defined on some distance-regular 
graph, with local constraints imposed at the vertices of the 
graph. Presently it is understood that the setting most amenable 
to analysis is that of codes defined on a regular bipartite 
graph. Ralf himself made an initial attempt to analyze such 
codes in a joint paper with Xiangyu Tang 1 11 1. The emphasis 
in ifTTl is on the estimation of the largest channel error rate 
tolerated by graph codes under such decoding. In the present 
paper, similarly to ifTOll , lfT3l and later works, the local codes 
are decoded up to their correction radius guaranteed by the 
minimum distance. 

II. Code ensembles 

An [N, K] binary linear code is a linear subspace of {0, 1}^ 
of dimension K. To construct an [N, RN] binary linear graph 



code C, consider an n-regular bipartite graph G{V = Vi LI 
V2,E), where the set of vertices V consists of two disjoint 
parts Vi , V2 of size m each, all the edges are of the form 
(w, u), u G Vi, u G V2, and the degree of every vertex v inV is 
n. Let A[n, Rqu, do] be a linear binary code of length n called 
the local code below. We identify the coordinates of C with 
the set E and for a vertex v E V denote by x{v) E {0, 1}" 
the projection of a vector x E {0, 1}^, N — nm, on the edges 
incident to v. A graph code C{G) is defined as follows: 

C={xE {0, 1}^ : Vvevxiv) E A}. (1) 

The ensemble of codes '^^{A, m) is constructed by associating 
a code C{G) with a graph G sampled from the set of 
graphs defined by a random permutation on N elements which 
establishes how the edges originating in Vi are connected to 
the vertices in V2. 

Generalizing this construction, consider an Z-partite n- 
regular uniform hypergraph H = (V, E) i.e., a finite set 
V = Vi U • • • U Vi, where \Vi\ = ■ ■ ■ = \Vi\ = m, and a 
collection E of /-subsets (hyperedges) of V such that every 
e E E intersects each Vi,l < i < Ihy exactly one element and 
each vertex v eV appears in exactly n different subsets of E. 
Aiming at constructing an [N, RN] binary linear code G by 
imposing local constraints at the vertices, we again identify the 
coordinates of C with the (hyper)edges of H. By definition, 
the code G is formed of the vectors x that satisfy condition ([TJ 
for every vertex in V. The ensemble of codes J^{A, l,m) in 
this case is constructed by sampling a random hypergraph from 
the set of hypergraphs defined by / ~ 1 independent random 
permutations on N elements. For i = 1,2, . . . ,1 — 1, the ith 
permutation accounts for the placement of edges between parts 
Vi and Vi+i of H. Of course, J^{A, I, m) becomes ^^(A, m) 
for I = 2. 

The following is known about the parameters of codes in 
the graph and hypergraph ensembles considered here. It is 
easy to see that the rate R of the codes G E M'^A, I, m) 
satisfies R > IRq - (l - = 2,3, . . . . Denote by d{Jf) = 
d{J^{A, I, m)) the average value of the minimum distance of 
codes in the hypergraph ensemble and let 

<5 = (5(^) ^liminf (2) 

A way to bound the value of 6 below using the distribution 
of distances in the local code A was suggested in ||5], ||8]. 
More explicit results in this direction were obtained in fT\, f3\. 
In particular, |1| shows that S{J^) > if the local distance 
do satisfies do > — 1). For the bipartite graph ensemble 
'i^{A,m) (i.e., for I = 2) this implies that do > 3, i.e., 
with high probability codes in ensemble are asymptotically 
good (have nonvanishing rate and relative distance) when the 
local codes correct one or more errors. For hypergraphs with 
/ = 3 or more parts any local codes (without repeated vectors) 
account for an asymptotically good ensemble. An explicit 
lower bound for 5{,^) that depends only on I and dp is given 
by [l ], see Theorem 15 .5 1 below. For the case when n is large 
and do = ^o^, a lower estimate of 5{,j^) is given by the 
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solution for x of the following equation |[T1 Cor. 6]: 

X I — 1 do 

Finally, if the local codes are chosen randomly as opposed 
to a fixed code A used at every vertex of H, then the codes 
in the (hyper)graph ensemble match the best known linear 
codes, i.e., reach the asymptotic Gilbert- Varshamov bound on 
the minimum distance JT]. 

Remarks. 1 . An equivalent description of the bipartite code 
ensemble is obtained by considering an edge-vertex incidence 
graph of the graph G{V, E), i.e., a bipartite graph D = {Di U 
D2,E) where Di = E, D2 = Vi L) V2, each vertex in Di is 
connected to one vertex in Vi and to one vertex in V2, and 
there are no other edges in E. Thus, for all v ^ Di, deg(u) = 
2 and for all v ^ D2, deg(w) = n. The local code constrains 
are imposed on the vertices in D2- By increasing the number 
of parts in D2 from two to I, we then obtain the hypergraph 
codes defined above. This gives an alternate description of the 
hypergraph code presented in Fig. [T] 

The ensemble of hypergraph codes with local constraints 
given by single parity-check codes was introduced by Gallager 
p, p. 12]. The proportion of errors correctable with these codes 
using the so-called "flipping" algorithm was estimated in ifTSl . 
Several generalizations of this ensemble were studied in IH, 

m. 

2. The derivations of this paper are not specific to binary 
codes: any local linear codes such as Reed-Solomon codes can 
be used in the construction with no conceptual changes to the 
analysis and the conclusions. 

III. Decoding algorithms for graph (Generalized 

LDPC) CODES 
Even though the ensemble m) forms a particular case 
of the ensemble J(f{A,l,m), in our analysis we employ 
different decoding algorithms for the cases / = 2 and / > 3. 
The reason for this is that edge-oriented procedures commonly 
used for bipartite-graph codes do not generalize well to 
hypergraphs. 

A. Decoding for the ensemble ^^(A, m). In our estimates of 
the number of correctable errors for the ensemble we rely 
upon the algorithm of ifTSll which iterates between decoding 
all the vertices in parts Vi and V2 in parallel using some 
decoding algorithm of the code A. Let C G '^{A^m) be a 
code. For the ease of analysis we assume that the local codes 
are decoded to correct up to t errors, where i > is an integer 
that satisfies 2i + 1 < o?o and do is the distance of the code A. 
Formally, define a mapping ipA.t ■ {0,1}" {0,1}" such 
that tl)A,t{z) — X G A if a; is the unique codeword that 
satisfies d{z,x) < t and tpA,t{z) = ^ otherwise. Let y'*) be 
the estimate of the transmitted vector before the ith iteration, 
i > 1, where y = y'^^ is the received vector. The next steps 
are repeated for a certain number of iterations. 

Algorithm I {y^^'^) 

• i odd: for all v e Vi put y(^+i)(ti) = '0A,t(y^'H^)); 

• i even: for all t; G V2 put = ipA^tiu^^H'^))- 

B. Decoding for the ensemble J^/f(A,l,rn). For the hyper- 
graph ensemble J^{A,l,m) we use the decoding algorithm 



proposed in |[T1. It proves to be the best choice in terms of 
the number of correctable errors among several possible algo- 
rithms for these codes such as the one in [4J and procedures 
analogous to Algorithm I above. 

Let C e .y/f{A,l,m) be a code and let H{V,E),V = 
Vi U • • • U V/ be the graph associated with it. For every 
« = 1, 2 . . . , / we will define an z-th subprocedure that decodes 
the local code A on every vertex in the part Vi. Suppose that 
a vector u G {0, 1}^ is associated with the edges e ^ E. 
Let fi.i, . . . , Wi,m be the vertices in the part Vi of H and 
let UiA — u{vi^i), . . . ,Ui^,n — u{vi^rn) be the TO subvectors 
obtained from u upon permuting its coordinates according to 
the order of edges in Vi and projecting it on the vertices 
Vi.i; • • • , Vi^m. In other words, the vector (iti.i, . . . , 
is obtained from u using the permutation that establishes 
edge connections between parts Vi and Vi. The ith subpro- 
cedure replaces the vector (m^ i, . . . , u.i,„i) with the vector 

(l/'yl,t(Mi,l)' ■ ■ • : V'A,t(Mj,m))- 

The algorithm proceeds in iterations. Let y G {0, 1}^ be 
the received vector Denote by F^^^ the set of estimates of 
the transmitted codeword (i.e., the set of iV-vectors) stored 
at the vertices of H before the jih iteration j = 1,2,.... 
After each iteration, this set is formed as the union of the 
vectors obtained upon decoding of the vertices in the ith part, 
i = 1, . . . ,1. Decoding begins with setting F*^^' = {y}. After 
the first iteration we obtain I potentially different vectors (one 
for each subprocedure) which form the current estimates of 
the transmitted vector. These vectors form the sets f/'^' , i = 
1, . . . ,1. In the next iteration each subprocedure will have to 
be applied to each of the I outcomes of the preceding iteration. 
Proceeding in this way, we observe that < 

This algorithm, called Algorithm II below, will only be 
applied for a constant number s of iterations until we can 
guarantee that at least one subprocedure has reduced the 
number of errors to a specified proportion, say from •joN 
to some 71 N, 71 < 70 . We then let another algorithm take 
over and decode all the candidates. Any low-complexity 
decoder of graph codes that removes an arbitrarily small 
positive fraction of errors 71 will do at this stage. This is 
because taking the proportion of errors from 70 to 71 > 
can be accomplished in a constant number s of steps, so the 
number of candidates that this decoder has to handle is at most 
P and does not depend on N. 

For the case of local codes correcting t > 2 errors we 
let this algorithm to be the decoding algorithm of bipartite- 
graph codes (Algorithm I), making sure that 71 is below 
the proportion of errors that are necessarily corrected by 
this algorithm for the ensemble ^^(A, m). This is possible 
because, leaving any two parts of the original hypergraph H 
to form a bipartite graph G, we obtain a random code from 
the ensemble $^(v4, m) which with high probability (over the 
ensemble) will remove all the residual errors from at least 
one candidate estimate. For t = 1 this approach fails for 
the reasons discussed in the next section, so we resort to a 
procedure in IT41 that corrects a small linear fraction of errors 
for single-error-correcting Hamming codes. 

Upon performing the described procedure we obtain a list 
of at most P candidate codewords of the code C. The final 
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Fig. 1. Alternate construction of tlie liypergrapli code: Tlie set Di = {ei, . . . , e]v}, where deg(ei) = I for all i, represents the coordinates of the code 
(hyperedges of H); the sets Vi, . . . , V;, where \Vj\ = m for all j, represent the vertices of the hypergraph H. Each vertex Vij, l<j<i, l<i<m 
carries a codeword of the local code A of length n. 



decoding result is found by choosing the codeword from this 
list closest to y by the Hamming distance. 

Though the last step of the decoding algorithm described is 
different from |[T1, the main idea is similar to that paper, so 
we refer to it for a more detailed description and a discussion 
of the algorithm. 

IV. Number of correctable errors for the 

ENSEMBLE ^(^,m) 

Let C e be a code and let G{V,E) be the graph 

associated with it. For a given subset of vertices S C Vi,i = 
1,2 and a vertex v denote by degg(w) the number of edges 
between v and S. Let Tr{S) = {v G V : ciegs(?;) > r + 1}, 
where r 6 {0, . . . , n — 1} is an integer. 

Below h{z) denotes the entropy of the probability vector 
z £ R"+^. In the particular case of n = 1 we write h{z) 
instead of h{z, 1 — z). 

Let t > he any integer such that 2t + 1 < do. The 
calculation in this section is based on the following simple 
observation. 

Proposition 4.1 : Suppose that for all 5 C Vi,i = 
1,2,15*1 < am, a G (0,1), there exists e > such that 
|T't(S')| < |5| — em. Then any atm — at{N/n) errors will be 
corrected by Algorithm I in O(logTO) iterations. 

Proof: Suppose that no more than atm errors occurred 
in the channel. Let Si be the set of vertices that are decoded 
incorrectly in iteration i of Algorithm I. The assumption of the 
proposition implies that \Si+i\ < |5i|(l — e/cr), so O(logm) 
iterations suffice to remove all the errors. ■ 
Define 



a\ 



where x > is found from the equation 

E E {"M"){<^{n-J)-^{l-a))x^^=-'-' ^Q. (5) 
i=0 j=t+i ' 

Let Zn = {z G [0,l]"+i : Er=o = 1} be the (n + 1)- 
dimensional probability simplex. 

The main result of this section is given by the next theorem. 



Theorem 4.2: Let A[n, Rqu, do] be the local code, let m 
oo, and let 2 < i < do/2. All codes in the ensemble '^^{A, m) 
except for an exponentially small (in N) proportion of them 
correct any combination of errors of weight atm in O(logm) 
iterations of Algorithm I, where Q < a < ao and ao is the 
smallest positive root of the equation 



l)h{a). 



Remark. The case of local codes with < = 1 is excluded from 
this theorem because G with high probability contains a large 
number of 4-cycles, which means that correcting single error 
at every vertex does not ensure overall convergence of the 
decoding. Indeed, if two vertices are affected by two errors 
each, and the corresponding 4 edges form a cycle, then the 
decoder will loop indefinitely without approaching the correct 
decision. The theorem is still valid in this case, but gives ao = 
0. 

Proof: We need to verify the assumption of Proposition 

l4Tn Let S C Vi,|5| = am and let mi = \{v e V2 : 
<legg{v) — — 1, . . . ,n. Clearly, 



mi < m, 



E^ 

i=t+l 



\Tt{S)\ 



\S\n. 



Let us compute the probability (over the choice of G) that 
\Tt{S)\ > {a — e)m. Let fj, = (mi, . . . ,rn„) be a vector with 
nonnegative integer components, let 

71 n 

M^{t, a) = {fi : mi < m, imi = aN, 

1=1 1=1 

n 

mi > (cr — e)^}, 

and let (™) denote the number of choices of subsets of size 
mi , . . . , m„ out of a set of size m. We have 



p m 



K5)|>|5|-6m) = ^ ^ 



n / \ m.i 

m \ 1 r / n ^ 



n 



(6) 

Let (s) denote the event that Vi contains a subset S", jS"! = s 
for which \Tt{S)\ > \S\ - em. We have 



P{^i{am)) < ]P{\Tt{S)\ > \S\ - em) 

\ am / 
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and 



P(|j^i(i)) <mP{J^i{(jm)). 



Denote by ^2 (f) an analogous event with respect to V2 . Then 

Verm/ 



p 



(U(^iWu^2W) 



< 



E 



\crNJ neM,(t,cr) 



n 



(7) 

Letting L to be the logarithm of the left-hand side divided 
by m and omitting Om(l) terms, we obtain the estimate L < 
n-'^Fn.t{(j), where 

FnA'^) = -{n-l)h{a)+ max (h{z)+Y^ z.log f")), 

i—1 

where 



i=l 



i=t+l 



and Zi — rrii/m, zq — (to — ^ mi)/m. 

The rest of the proof is concerned with the evaluation of 
the above maximum. Define 



g{z) = h{z) + ^ Zi log! ^. 
a = sup{(T > : Fn,t{y) < for all < y < a}. 



(8) 



As long as cr < a, the probability of not being able to 
correct atm errors with a random code from the considered 
ensemble approaches zero. Thus, we need to find the maxi- 
mum max2£^/(t cr) g{z) for all a € [0, a). The proof will be 
accomplished in the next three steps. Since e will be assumed 
arbitrarily small, we will omit it from our considerations and 
write instead of ^g'. 

1. We find the point z* that gives the maximum of g{z) 
without the constraint X^ILt+i — 

2. Next we show that for < u < ct, the point z* ^ 
and therefore the maximum over is attained on the 

boundary, i.e., we can replace with 



n 

I, E = 



'i=t+i 



3. Finally we compute the value of the maximum. 

Step 1. Without the constraint X]"=t+i '^1 ^ ^ '^he maximum 
is easily computed. Indeed, the proportion of edges incident 
to the vertices in S out of the edges of G is cr, so the 
fraction of vertices with ^-degree i should be close to z*{a) — 
(")(t'(1 — cr)"^*. Thus, the coordinates of the maximizing 
point z* = z*{a) are z*,i = 1, . . . , n; zq = 1 — J^i : ™d 

g{z*) = nh{a). 

Slightly more formally, note that z* is the unique stationary 
point of the function g{z), and that this function is strictly 
concave in z. Therefore, z* is a unique maximum of g{z) on 
Zn, and the function g{z) grows in the direction z* — z for 
any 2; e Z„. 

Step 2. Suppose that < ct < cr. Observe that p(cr) = 
'nl=t+i ^ P{X > t+l), where X is a (cr, 1— cr) binomial 



random variable. This probability is monotone increasing on 
cr for cr e [0, 1], and p(0) = p'{0) = 0. Thus for cr e [0,a) 
where a is the smallest positive root of X]"=t+i ^ii'^) — 
we have 



E < - E :v^(i 



and so the point z*{a) ^ ^'{t,a). Our claim will follow 
if we show that a < a. This is indeed the case because for 

< CT < CT, 

max g{z*{a)) < {n - l)h{a). 

On the other hand, g{z*{a)) = nh{a). This establishes that 
the maximum of g{z) on z G is attained on the hyperplane 

En 
,:=t+l 2. = CT. 

Step 3. To compute the maximum of g{z) on z, let us form 
the Lagrangian 



U{z,Ti,T2) = h{z) + ^Z,l0g f " 
i=l V* 
n 

+ Ti 1^ ^ izi - arij + T2 ^ Zi - a 

2=1 j = t+l 

Setting VC/ = and ri logx, T2 ~ logy, we find that 




xW ifO<i<t 



yx^D \f t < i < n, 



where we have denoted 



D 



i=0 ^ ' i=t+l 

Adding these equations together, we find conditions for x and 

n / \ 



i=t+i 



X 



cm = D 



z 



y E * 

i=t+l 



Once y is eliminated from the last two equations, we obtain 
the condition ^ for x. Finally, substituting the found values of 
Zi,i = 1, . . . ,n into g{z), we find that the maximum evaluates 
to the expression F„_t(cr) given in (01 (and therefore, a = ctq). 
Since we seek to obtain a value L < 0, the boundary condition 
for the proportion of correctable errors is obtained by setting 
L = 0. This concludes the proof. ■ 

Example 1: Using Theorem 14.21 together with ^ we can 
compute the proportion of errors corrected by codes in the 
ensemble '^^{A,m),m 00 for several choices of the local 
code A. For instance, taking A to be the binary Golay code 
of length n = 23 we find ctq w 0.0048586 and therefore, the 
proportion of correctable errors is ^ w 0.00063. Similarly, 
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for the 2-error-correcting [n — 31, k — 21] BCH code we find 
o-o « 0.000035 and ^ w 0.0000023. 

To underscore similarities with the results obtained for 
product codes and their later variations including graph codes 
(e.g., ifTSl ) we compute the proportion of errors correctable 
with codes from the ensemble m) in the case of large 
n. 

Proposition 4.3: Let t = rn. Then the ensemble ^^(v4, m) 
contains codes that correct cjtN errors for any cr < cto, where 
(To is given by 

'x{l - t)^ 



CTq = sup 



{^>0:Vo<.<. (l-a;)/.(^^-^) 
+xft(r) + En < h{x)^ 



where £„ = (1 + logn)/n. 

Proof: Referring to the notation of the previous proof, let 
us evaluate the asymptotic behavior of the exponent L of the 
probability in (|7]i. Since h{z) < \ogn, we have 



W^Fn^ticr) < -h{a)+n- 



E 



Zi log 



i=0 

+ n^^{l + logn). 



Next, 



i=0 



i=0 



i=t+l 



V (1 — (Tjn/ \ an 



Let y — n ^ X)"=t+i '^^en for any z G ^{rn, a) we have 



1 

- z. Ids 



< max 

I j rcr<y<a 



{(i-)'.(H)-Kf)}- 



The function on the right-hand side of this inequality is 
concave. Its global maximum equals h{a) and is attained for 
y = a"^ . Thus, assuming that cr < r, we conclude that the 
constrained maximum occurs for y = rcr, which gives the 
following bound on n^^Fn.t{<j) ■ 

n'^F^^tia) < ^h{a) + (1 - a)h(^^^—^) + ah{T) + £„. 



1 - cr 

As long as the right-hand side of the this inequality is negative, 
the previous proof implies that the code corrects all errors of 
multiplicity up to arN. ■ 
From the expression of this proposition we observe that (as 
n oo) the value of do approaches r, so the ensemble 
contains codes that correct up to a proportion of errors, 
where rn = do/ 2 is the error-correcting capability of the 
code A. This result parallels the product bound on the error- 
correcting radius of direct product codes. As in the case 
of product and expander codes (e.g., [2j), the proportion of 
correctable errors can be improved from = (do/(2n))^ by 
using a more powerful decoding algorithm. 



V. Number of correctable errors for the 

ENSEMBLE J^{A, I, m) 

In this section we first state a sufficient condition for the 
existence of at least one subprocedure within each step of 
Algorithm II that reduces the number of errors, and then 
perform the analysis of random hypergraphs to show that with 
high probability this condition is satisfied. Overall this will 
show that the number of errors in at least one of the candidates 
in the list generated after a few iterations is reduced to a 
desired level. 

Denote by E{v) the set of edges incident to a vertex v e 
V. Let C G ^(A,Z,m) be a code and let H{V,E) be its 
associated graph. Let E <Z E he the set of errors at the start 
of some iteration of the algorithm. The next set of arguments 
will refer to this iteration. Let d ^ {v eVi : \E{v)r\E\ < t} 
be the set of vertices such that each of them is incident to no 
more than t edges from £ (such errors will be corrected upon 
one decoding). Let Bi = {v € Vi : \E{v) D £\ > do — t} he 
the set of vertices that can introduce errors after one decoding 
iteration. Note that each of such vertices introduces at most t 
errors. 

The main condition for successful decoding is given in the 
next lemma. 

Lemma 5.1: Assume that for every £ d E,\£\ < there 
exists i = i{£), I < i < I such that |f (Gi)| > t\B^\ + eN, 
where £{Gi) is the set of edges of £ incident to the vertices 
of Gi and e > 0. Then for any < /3 < 7, Algorithm II will 
reduce any errors in the received vector to at most jiN 
errors in c(/3, 7, e), iterations where c is a constant independent 
of TV. 

Proof: We need to prove that at least one of the sub- 
procedures will find a vector with no more than /37V errors 
after a constant number of iterations. In any given iteration by 
the assumption of the lemma there exists a component Vi for 
which the ith subprocedure will decrease the count of errors 
by |f (Gi)! — tBi > eN. Thus, in each iteration there exists a 
subprocedure that reduces the number of errors by a positive 
fraction. ■ 
Next we show that the assumption of Lemma ISTI holds with 
high probability over the ensemble. Consider the function 

where in this section the region ^(^,7) will be as follows: 



n in 



^ iZi = 7n, ^ 

i—\ i—\ i—do — t 



(9) 



Lemma 5.2: Let m 00 and let 

70 = sup{a; > : Vo<^<:r {l/n)Fn,t{l) < {I ~ l)/i(7)}- 

(10) 

A hypergraph from the ensemble of Z -partite uniform n-regular 
hypergraphs with probability 1 — 2^^(^) has the property that 
for all £ C £',1^1 < 70^, and some e > 0, the inequality 
\£{Gi)\ > t\B.,\ + eN holds for at least one 



7 



Proof: Let £ C E,\£\ = 77V. Let m, = \{v G 
|£;(f)nf| = = Clearly |£(Gi)| = ELo ' 

and \Bi \ = Yl'i=do-t'^i- have 

P^P{\£{G,)\ <t\B,\+eN) 



m \ 1 r / n 



17^/ n£M^(t,i) i=0 



where = {mi, . . . ,m„}, 

n 

M,{t, 7) = {m e (Z+ U 0)" : ^ TOi < m, 

n t n 

i— 1 i—1 i—d(j — t 

Denote by ^{£) the event that for a given subset £ C 
E,\£\ = no part Vi of H satisfies the assumption of 
Lemma O Then P{^{£)) = and 

P{3£ : {\£\< 7iV) A {^{£))} < 

Letting L to be the logarithm of the left-hand side of this 
inequality divided by N and omitting ojv(l) terms, we obtain 

i < -(/ - l)/i(7) + - max g{z), (11) 

where g{z) is defined in (O, 

n t n 

■y^'{t, 1) = {z ^ Zn ■■ ^ izi = 7^, ^ < ^ izi} 

2—1 2—1 i—do—t 

and = mi/m (as in the previous section, we have omitted 
e which can be made arbitrarily small). 

The proof will be complete if we show that the opti- 
mization region can be replaced by ./#. For that we 
follow the logic of the second part of the proof of Theorem 
14.21 As before, the maximum of g{z) without the constraint 
Y^l^i^Zi < J27=do-t^^i attained at the point 2;* (7) = 
••■,<) G -Zn, where 

We need to show that as long as < 7 < 70, the point 
z* ^ ^'{t,'y). By concavity of the objective function and 
the optimization region, this will imply that the maximum is 
on the boundary. As before, it is possible to show that in the 
neighborhood of 7 = 0, 



J2 > 



i—l i—d() — t 

and thus for ^ < j3, where p is the smallest positive root of 
E -=1 = Er=do-t the point z*{^) ^ .^'{t, 7). Let 

7 = sup{7 : VO < a; < 7, rhs of dTTJ < 0}. 

We note that for all 7 < 7, 

max g{z) < {I — 1)71/1(7). 



On the other hand, g{z*{f3)) = nh{f3). This implies that 7 < 
/3, and so for all 7 < 7, the point z*{'^) ^ .^'(^,7). Thus 
the region ./#' in the maximization can be replaced with ^ 
(and 7 = 70). ■ 

This lemma establishes that the number of errors in at least 
one of the candidates in the list generated after a few iterations 
is reduced to a desired level. After that the residual errors can 
be removed by another procedure as described above. In this 
situation we say that the errors are correctable by Algorithm 
II, without explicitly mentioning the second stage. 

In the next theorem, which is the main result of this section, 
5 refers to the lower estimate of the average relative distance 
of the hypergraph code ensemble ^ from Theorem l5.5l below. 

Theorem 5.3: Let t > 2 be the number of errors correctable 
by the local code A. Algorithm II corrects any combination of 
up to A^(min(7o, (5/2)) errors for any code C £ ,J^{A,l,ni) 
except for a proportion of codes that declines exponentially 
with the code length N = nm, m cxo. 

Proof: With high probability over the ensemble of hyper- 
graphs considered, for a given hypergraph H{V, E) a constant 
number s of iterations of the algorithm will decrease the 
weight of error from 70 to any given positive proportion 
j3 for at least one of the candidates in the list Y^'^ . Take 
/3 = (To: where (Tq is the quantity given by Theorem 14.21 
Next consider the bipartite graph G{Vg — V^i Ul^, Eq) where 
Vi , V2 are the parts of H and where (wi , W2 ) G Eq if ?; 1 , W2 G e 
for some edge e £ E. By the previous section, with high 
probability these <to^ errors can be corrected with O(logm) 
iterations of Algorithm I. Finally, the correct codevector will 
be selected from the list of candidates because the proportion 
of errors is assumed not to exceed NS/2. ■ 

The complexity of this decoding is O(A^logA^) where the 
implicit constant depends on the code A. 

In the following theorem we extend the results of this 
section to the case of A being a perfect single-error correcting 
Hamming code of length n = 2*" — 1 for some r — 3,4,.... 
In this case the maximum on z in the above proof can be 
computed in a closed form. As remarked above, in this case 
in the last part of the error correction procedure we use the 
decoding algorithm of [ 14] to remove residual errors from the 
candidate vectors. 

Theorem 5.4: Suppose that the local codes A are taken to 
be one-error-correcting Hamming codes and let 6 ~ 5{,yf) be 
the relative average distance (|2]l of the ensemble (A, l,m). 
Then almost all codes in the ensemble J^f{A,l,m) can be 
decoded to correct A^min(7o, (5/2) errors, where 70 is given 
by (doll and 



F„sh) = -7nloga; + log 1 + 2 



A 



E 



(12) 



where x is the only positive root of the equation 
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Proof: It is obtained by maximizing the function g{z) 
over the region 

n n 

^(1, 7) = {2; e Z„ : ^ = 7n, 2^1 = ^ ^^J- 

i=l i=2 

The Lagrangian takes the form 

n / \ ^ 

/i(2;) +^z.(logn + log f ^ n + A(^(i + l)z, -771), 

where 2; = (2:1,^2, ■■■,ZnA~Y.i ^i) -^1 = Ei"=2 
A is an arbitrary muhipHer. Setting the partial derivatives to 
zero, we find the value A to satisfy 2^ — A, where x is given 
above. The calculations are tedious but straightforward and 
will be omitted. ■ 

The last theorem enables us to find the proportion of 
correctable errors for the case when A is the Hamming code 
of length n ^ 2^ — l,t ^ 1. Since the examples below rely 
on the value of the ensemble-average distance, we quote the 
corresponding result from 11]. 

Theorem 5.5: IT] Thm.5] Let 5{M') be the asymptotic av- 
erage relative distance of codes in the Z-hypergraph ensemble 
constructed from the local code A of length n and distance 
do- Then 

6{jr) > supfc. : llogil^i^i^silM < il-l)hiu;)} 
where xq = a;o(a;) is the positive solution of the equation 




For instance, for the case 71 = 31, / = 5 this theorem gives 
the value of the relative distance 6(,y^) > 0.01618 (the rate 
of codes R > 6/31). Performing the calculation in ( fT2] i. we 
find that the average code from the ensemble J^{A, 5, m) the 
proportion of errors correctable by codes in the ensemble using 
Algorithm II to be at least 70 = 1.2 x 10^^. 

We include some more examples. In the following table 



Example 2: 



I 


17 


23 


28 


34 


Rate 


0.7006 


0.5949 


0.5069 


0.4012 


70 


0.000235 


0.000401 


0.000521 


0.000644 




0.00415 


0.00504 


0.00558 


0.00608 


I 


40 


45 


51 




Rate 


0.2955 


0.2074 


0.1018 


70 


0.000747 


0.000821 


0.000898 




0.00648 


0.00676 


0.00704 



It is also of interest to compute the values of 70 for code 
rate i?(C) w 0.5. 



n 


127 


255 


511 


1023 


I 


9 


16 


28 


51 


Rate 


0.5039 


0.4980 


0.5068 


0.5015 


70 


0.0002012 


0.0004873 


0.0005207 


0.0004227 




0.01157 


0.008658 


0.005581 


0.003394 



These estimates are at least an order of magnitude better 
than the corresponding results in ||6|, llT4l obtained for LDPC 



codes and their generalizations based on the "flipping" algo- 
rithm of {I5j. 

The case of large n. As in the previous section, it is 
interesting to examine the case of long local codes A because 
it reveals some parallels with the analysis of the decoding 
algorithm in the case of nonrandom hypergraphs [ 1 J . We begin 
with the observation that the proportion 70 of correctable 
errors for the ensemble Jf{A, t, m) computed above is a 
function of the number of errors t that each local code corrects 
in each iteration. 

Lemma 5.6: Let t = rn, do = Squ. The ensemble 
Jif{A,t,m) contains codes that correct •jN errors for any 
7 < 7o(''') — min(T, xo(t)) where 

/ X \ / XT \ X 

xo(t) ^sup{x > : (l-^)h[- +—h{5o-T) 

+ e„ < {l^l/l)h{x)} 

and e„ = logn/n. 

Proof: Referring to the proof of Lemma 15.21 we aim at 
establishing conditions for the exponent L of the event -Sf ) 
to be negative as m approaches infinity. We assume that 7 < r 
(otherwise our estimates do not imply that the convergence 
condition of Lemma 15.11 holds with high probability over the 
graph ensemble). 

From ( fTTT ). ^ we have 

n . , , 

L < -{I - l)hi^) + I max yz,h(-)+ -^^, 

z=0 

where ^(t, 7) is defined in (|9]l. Next, write 

where we have denoted X]i=o -^^ ~ ^' X]i=i = Mi'^- I" 
addition let us put 'YTi=da-t ~ M2'^j then the values of 
the sums zi and izi over each of the three intervals 
Ji = [0, t], /2 — [t + 1, do — t — 1], /a = [do — n] can be 
found from the following table: 

h h h 

E Ml 7 - Ml - M2 M2- 

The variables introduced above depend on the point z and 
satisfy the following natural constraints: for any z G ^(t, 7), 

Ml ^ '''^ 

r{l-X-^)< 7-Mi - M2 < (<5o - t)(i - A - ^) 

{So-r)^<l^2<^. (14) 

T T 

Proceeding as in JTSl l, we can estimate the sum on in L as 
follows: 

n 

< /(A,m,M2) (15) 

i=0 
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where 



+ 1-A 



7 - /^l - /i2 

-A - (Mi/r) 



Ml 



Our plan is to prove that some of the inequalities in (fT4l l 
can be replaced by equalities, thereby expressing the variables 
A, /xi, /i2 as functions of 7. r. We will rely on the fact that the 
function / is concave in its domain, proved in the end of this 
section. 

Note that for all 2; e Z„ the sum 



■IL 

^z./.(l)<M7) 



1=0 



and that it equals h{'-f) at the point z such that Zi = 1 for 
i = \jn] and Zi = elsewhere. Also note that since 7 < r, the 
point z is outside the region ^{t, 7) and thus, by concavity. 



Let be the point at which this maximum is attained, and let 
Xi = (A, ^i, 1JL2) be the corresponding point for the arguments 
of /. By construction, the point Xi satisfies the inequalities of 
(IT4] |. At the same time, consider the function /(■) on the line 
A = /zi = ^2- As the variables approach along this line, the 
value /(A,/ii,/i2) approaches ^.(7). 

To summarize, we have found two points, Xi and X2 ~ 
(0, 0, 0) that are located on different sides of the hyperplane 



T 1 



A-^ 



7-^1-/^2 



such that f{xi) > a,f{x2) > a. Invoking concavity of the 
function /, we now conclude that there is a feasible point x' 
on this hyperplane such that ,f{x') > a. 

Therefore, put /i2 = 7 ^ t{1 — A) and write 



/i(A,Mi 



1 - A 



T \ /il 



where the variables are constrained as follows: for any z £ 

^(t,7), 



Ml ^ ■''•A 
t(1-A)-;Ui >0 

(^o-r)^<7-r(l-A)<^ 

T T 



(16) 
(17) 



Since /i is a restriction of / to a hyperplane, it is still concave. 
Now notice that r) = h{'j) and that the point (1, r) does 
not satisfy inequality ( fTSI l and the left of the inequalities ( fTTI l. 
Repeating the above argument, we claim that the function / in 
( fTSl l can be further restricted to the intersection of the planes 
r(l — A) — /il and {Sq — T){fii/T) = 7 — t(1 — A). Altogether 
this gives: 

A = 1 - 7/^0, Ml = Jt/Sq. 



Let us substitute these values into the expression for fi and 
rewrite ( fTSl l as follows: for any < 7 < r, 



max 



n 

^ Vn/ V do/ V(5o-7/ Oq 

1=0 

(18) 

Thus if the condition in the statement is fulfilled then i < 0. 
This concludes the proof. ■ 

Remark. The main part of the proof is estimating the 
solution of the following linear program 

n 

1=1 

z = (zo, 21, . . . , 2„) e ^{t, 7) 

where the variables define a probability distribution on 
{0,1,..., n}. It is clear from concavity that the maximum 
is attained at the point where among all the indices i G /i at 
most one value is nonzero, and the same applies to I2 and 
I3 . We have shown that the value of the program is bounded 
above by the right-hand side of ( fTSl ). The following point gives 
this value and is therefore a maximizing point: 



1 



2_ 



7 

= T-, Zi = otherwise, 



where ii = njr/ {60 — 7), 12 ~ n{SQ ~ t). Since 

7r 

do - 7 

this shows that the worst-case allocation of errors to vertices 
in a given part of the graph assigns no edges to vertices that 
are neither good nor bad. This also confirms the intuition 
suggested by Lemma 15.11 that bad vertices (vertices assumed 
to add errors) should each be assigned the smallest possible 
number of error edges do — t. 

The next proposition is now immediate. 

Proposition 5.7: The ensemble Jf{A, I, m) with long local 
codes contains codes that can be decoded using Algorithm II 
to correct all error patterns whose weight is less than 70 A'^, 
where 

70 = max 70 (t). (19) 

0<r<5o/2 

Estimating the number of correctable errors for the ensem- 
ble Jff{A,l,m) from Proposition 15.71 analvticallv is difficult 
because it involves optimization on t (generally, the local 
codes should be used to correct a smaller than So/2 proportion 
of errors). We note that in the particular case of r = (5o/2 the 
proof of Lemma 15.61 can be considerably simplified, although 
the resulting value of 7 is not always optimal. 

Example 3. Let / = 3. Using local codes with Sq = 0.05 we 
can construct hypergraph codes of rate R > 0.19. From IT] 
Cor 6], the ensemble-average relative distance is at least S ~ 
0.0112 and the proportion of errors correctable by Algorithm 
II is found from ^ to be 70 « 0.0035. 

Example 4. Let So = 0.01 and I = 10. In this case, we find 
from 1 1 , Cor 6] the value of the relative distance S « 0.00599. 
The code rate satisfies R > 0.14. Performing the computations 
in fT% and Lemma 15.61 we find the estimate of the proportion 
of correctable errors to be 70 « 0.002198. 
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Proof that /(A,/ii,/i2) is concave. First we prove that the 
function 

cj,[x,y) = {l~x)h[:LZJL) 



is concave (not necessarily in the strict sense) for < x, y < 
1,0 < 7 — y < 1 — X. For that, let us compute its Hessian 
matrix: 



H 



1 ( (1— x)(7— iz+rc — 1) -y—y+x—l 



In 2 



l-x 



1-y+x-l h-v){i-y+x-i) / 
The eigenvalues of H are 

(7 - 2/)^ + (1 - x)2 



0, 



<0, 



(1 - x){-f - y){-f -y- (l-x)) 
so H ^ 0, and so 4> is concave. Next observe that the function 



T/ M-A-(mi/t). 
can be obtained from (p hy a linear change of variables 

x = X + ^i/t, y = Ml + M2 

and therefore is also concave. Finally, the functions A/i(/ii/A) 
and [ni / T)h{iJL2T / fLi) are also concave, and thus so is the 
function /(A, ^1,^2). 
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VI. Conclusion 

We have estimated the proportion of errors correctable by 
codes from ensembles defined by random Z-partite graphs, 
I > 2. In contrast to the case of expander codes tlOi . 1131 . 
im, flU, Q our calculations cover the case of local codes 
of arbitrary given length and distance, including small values 
of the distance. The behavior of code ensembles considered 
here was examined from a different perspective in |[T] where 
we computed estimates of the expected distance and weight 
distribution of these codes. The paper [ 1 1 and the present work 
together provide answers to the set of basic questions regarding 
random networks of short linear binary codes and extend our 
perspective of concatenated code constructions to the case of 
sparse regular graphs. 
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